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Q , Abstract 

\ The top flavour-changing neutral couplings can be large in extended models 

00 ' with vector-like quarks. In the next decade(s) the CERN Large Hadron Collider 

will allow to measure (bound) them with a precision of few per cent. 
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> : 

q\ '. 1 Introduction 

m ; 

The mixing among light quarks u, d, s, c, b is precisely measured and in agree- 
| ment with the Standard Model (SM) Q. In contrast the gauge couplings 

ON ' of the top quark are poorly known experimentally. However, its mixing 

r~| ■ is strongly constrained in the SM. In particular, the Glashow-Iliopoulos- 

Qh! Maiani (GIM) mechanism Q forbids all tree-level flavour-changing neutral 

O currents. On the other hand the top mixing can be large in simple SM ex- 

tensions. It is then important to measure (bound) it because any positive 
signal will stand for new physics. 
.^J | The top flavour-changing neutral couplings (FCNC) with a light quark 

^ ■ q = u,c and a Z boson, a photon A or a gluon G a are conveniently 

^ , parametrized by the Lagrangian || 



£ = -Y-i^(x t L q P L +X t R q P R )qZ^ 
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where Pl,r = 2 5 ano - are ^ ne Gell-Mann matrices normalized to fulfil 
Tr(T a T b ) = ^2~. These vertices are constrained by the data collected at the 
Fermilab Tevatron § 

X tq = y/\X%\> + \X*\* < 0.84, 
«t„ = v^fpTRfp < 0.78, 

s ^cS¥TKg¥<o.i5. 

At this point one can make three obvious questions: 

• How large can these couplings be when all experimental data are taken 
into account ? 

• Do simple models exist saturating the resulting limits ? 

• How precisely will future colliders measure these vertices ? 

We will address these three questions in turn. First we review the limits 
on the mixing between light and heavy quarks in Section 2, paying special 
attention to the t couplings. Then in Section 3 we argue that the top mixing 
can be large in the simplest SM extensions with vector-like quarks. Finally 
we discuss the precision with which these couplings will be measured at 
future hadron colliders in Section 4. Section 5 is devoted to conclusions. 
For an extended analysis see Ref. |!|. 

2 Experimental limits on quark mixing 

Eq. (|l]) is a piece of a general effective Lagrangian of which the SM is the 
lowest order part ||. All terms in C result from dimension six operators 
after spontaneous symmetry breaking. But it can be argued that having 
different origin the size of their coupling constants can be quite different. 
The 7^ terms are dimension four and we expect them to be larger than 
the terms which are dimension five. This is what happens in the mod- 
els we study in next Section. However, when dealing with future limits at 
large colliders in Section 4, we must investigate all possible scenarios and 
then allow for arbitrary a^ u couplings within existing experimental bounds. 
In the rest of this Section we discuss the limits on dimension four (renor- 
malizable) couplings. We include charged currents because as in the SM, 
charged {Cw) and neutral (Cz) currents are related in SM extensions with 
vector-like fermions. 
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2.1 Direct limits 



The W couplings V qq t in 



where the sum over q and q' is understood, are consistent with a 3 x 3 unitary 
Cabbibo-Kobayashi-Maskawa Q (CKM) matrix jjj], 

\V ud \ = 0.9740 ± 0.0010, \V US \ = 0.2196 ± 0.0023, \V ub \ = 0.0033 ± 0.0008, 
jv^] = 0.224 ± 0.016, \V CS \ = 1.04 ±0.16, jv^l = 0.0395 ± 0.0017. 

(4) 

While the Z couplings c ', in 

£z = (q L c L qql ^q' L + qRcfo-fJn) Z„, (5) 

where c^ q R are equal to xh R in Eq. (|l|), are diagonal and universal within 
experimental errors J?], ||, g], 

|<&*| < 1.2 x HT 3 , 



I (XL I ' 

|c^| < 4.1 x 10~ 5 , \c L d f\ < 1.1 x 10~ 3 , \c^ R \ < 1.9 x 10~ 3 , 
c L u = 0.656 ± 0.032, c£ = 0.690 ± 0.013, 



c « u = -0.358 ± 0.026, c£ = -0.321 ± 0.019, 
^ M = -0.880 ± 0.022, c£ b = -0.840 ± 0.005, 
4i = "0.0541°;^, ^ = 0.194 ± 0.018. 



(7) 



The diagonal u and d couplings are measured in atomic parity violation 
and in the SLAC polarized-electron experiments and the c and b couplings 
result from a fit to precise electroweak data at the Z peak |fTCf| . The sign 
of the right-handed (RH) couplings is fixed by the corresponding off-peak 
asymmetries. On the other hand the s couplings are less precisely measured 
but also consistent with the SM fermion assignments. The off-diagonal cou- 
plings in Eq. (^) vanish in the SM as a consequence of the GIM mechanism 
[0. Whereas the diagonal couplings in Eq. (|7|) are a function of a unique 
angle 

^ R = 2(r^ R -Q q sm 2 9 w ), (8) 

where T% q ,R and Q q are the quark isospin and electric charge and 9w the 
electroweak mixing angle. All data in Eqs. (UHf?]) agree with the SM within 
experimental errors, but what do we know about the top quark ? 



3 



At Fermilab Tevatron it has been established that 

\V tb ? 



|Vw| 2 + |Vk| 2 + |V«,| : 



0.99 ±0.29, (9) 



however, although consistent with the unitarity of the 3x3 CKM matrix, 
this ratio tells little about it. On the other hand (Eq. 



c 



% R \ = \X% R \ < 0.84, (10) 

if t mainly decays as in the SM. LEP2 data give a similar limit [^]. Thus, 
the direct bounds on top mixing, Eqs. (PJIoD, are somewhat weak, leaving 
room for large new effects near the electroweak scale. 

2.2 Indirect limits 

One must also wonder about indirect constraints on quark mixing, although 
they are model dependent. Let us revise the estimates discussed in Ref. 
[f[3f for illustration. Assuming only one non-zero FCNC c^ c in Eq. (|5|) and 
integrating the heavy top quark the following effective Lagrangian for the 
left-handed (LH) down quarks is generated: 

L ^ = -^^m^^ml ( V ^ Vti + ^ Va ) W^L^+h.c., (11) 

where A is a cut-off integral and v the electroweak vacuum expectation value 
~ 250 GeV. Then bounds on c\ c can be derived comparing with neutral me- 
son mass differences and rare decays. However, although with a minimal 
set of assumptions, this calculation is too rough and the corresponding in- 
direct constraints too stringent. Whatever new physics is beyond the SM 
is unlikely that the net effect is only one non-zero FCNC. In general there 
will be more heavy degrees of freedom which after integrating them out will 
generate new effective contributions cancelling partially C e J in Eq. (|Tl|). 
This is the case, for instance, in the simplest SM extension with a new quark 
isosinglet T of charge |. As discussed in next Section the V*jC^ t Vu + V t *c^ c V c i 
coupling in Eq. ( |T[| ) must be replaced (up to diagonal terms) by V*jX^gV^i, 
where a, /3 run over all quarks of charge | and X^ c = c\ c . As X^g = V am Vp m , 
where m = d, s, b is summed up, then 

V*jX^pV/3i = V*jV arn Vp m V8i = bjrabmi = (12) 
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implying no FCNC in the down sector if the up quarks are degenerate. 
When the actual couplings and masses are introduced, the cancellation is 
not complete. If T is very heavy, it decouples and t does not have large 
FCNC. 

The constraints on RH FCNC appear to be weaker due to the absence 
of RH charged currents in the SM. Still similar comments apply. If only one 
RH coupling in Eq. (||) does not vanish, the p parameter through the Z 
boson self-energy bounds its size. Again in definite models this restriction 
is smoothed. If the SM is extended with a new heavy quark isodoublet 
/ T 
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B 



of charges I \ J , the T and B contributions to the p parameter 

cancel if the quarks are degenerate. If the top mixing is non-negligible the 
cancellation is not complete. 

In summary, indirect constraints restrict FCNC but their application 
must be done in a model dependent basis. In fact in the two SM extensions 
just mentioned the top FCNC can be large but without saturating the direct 
bounds in Eq. (|io|). For example |J: 



\ct c \ = \Xt c \ < 0.082, for an extra quark isosinglet T, 
|c^| = \Xfo\ < 0.16, for an extra quark isodoublet 



T \ (13) 
B ' 



The mixing with the u quark can be almost as large as with the c quark but 
the top can not have a large mixing with both at the same time. Otherwise, 
it would be also a large coupling between u and c, what is experimentally 
excluded (Eq. (§)). 

3 Simple SM extensions with large top mixing 

Let us discuss in more detail the two simplest SM extensions with vector-like 
quarks allowing for a large top mixing with the up or charm quark. 

3.1 One extra isosinglet 

The charged and neutral current terms in the Lagrangian read (in the current 
eigenstate basis) 

Cw = "Sv^ 7 ^^ + hx ' (14) 



5 



and 

Lz = ~2^ ( fi «^ u « " ^< ~ 2s w J em) Z„ (15) 

respectively, i = 1,2,3 are the three standard families. Diagonalizing the 
fermion mass matrices, the current eigenstates can be written as linear com- 
binations of the mass eigenstates 

<i = ^V, d° Ll = U?fd Lj . (16) 

Greek (Latin) indices run always from 1 to 4 (3). Then 

C w = -^u La V a a»d Li W+ + h.c. (17) 

and 

Cz = ~2^ (uLaX^ULp - du-fdu ~ lsl?J% M ) Z M , (18) 

with 

V aj = U^*U^ (19) 

and 

as announced in Section 2 and used in Eq. (|T2]). (Note that we have 
introduced a superscript to distinguish up, u, and down, d, mixing ma- 
trices. We omit it when the indices specify the quark flavour, as for in- 
stance in Eq. (H).) This type of models have been studied by many authors 
|9], [bf| . The important point here is: how large can the top mixing be ? 
U^o parametrizes the departure from the SM. The constraints on V qq i and 
c qq^ = ^^qq^ ~ 25q 9 ' Q g sin 2 (where the +(— ) sign is for the up (down) 
quarks) from present data (Eqs. (@,^,@)) translate into (s^,c^ stand for 
sinus and cosinus of the corresponding mixing angle) 

UtL = (0 ss' sc f c) , (21) 

where the up entry is much smaller to maximize the top mixing with the 
charm quark X t L c = -s 2 s'c' in Eq. @), obtaining < 0.082 [|. If 

we want to maximize the top mixing with the up quark, the role of the 
charm and the up is reversed in Eq. (pl|), and \X^ U \ < 0.047. The indirect 
constraints are also fulfilled. 
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3.2 One extra isodoublet 



large. With an extra quark isodoublet ( the charged and neutral 



In this model there are RH charged currents and RH FCNC which can be 
current terms in the Lagrangian read {v? L ± (d^ 4 ) corresponds to T® (-B£)) 



and 



= ~2c^' u LaTU La + T R YT R - d La ^d La 



B° R ^B° R - 2s 2 w r EM ) Z M , (23) 
;e basis, and 

CW = ~2^2 (? LaV ^ dL P + tmV&fdRf,) W+ + h.c. (24) 



in the current eigenstate basis, and 
9 



and 

lew 

-d Ra X d a ^d m - 2s 2 w J£ M ) (25) 
in the mass eigenstate one. The generalized CKM matrices are written 

V£, = U£*U$, V$ t = U$ZU$ j3 (26) 
and the neutral couplings 

vuR ttuR*ttuR vdR TjdR*TjdR to t 7\ 

where the unitary matrices U L ^ diagonalize the quark mass matrices. A 
similar analysis as for the isosinglet gives 



U, 



uR 



(0 s c 0) , U$ a = (0 e Vl - e 2 ) , (28) 



with X^: = sc and \X^\ < 0.16 ||. On the other hand the measured 
value of b — > S7 implies |e| < 0.001. If the top mixing with the up quark is 
maximized, \X^\ < 0.14, fulfilling also the indirect constraints. 

Once we have shown that the mixing between the top and the up or 
charm quark can be large, we would also like to know how well can it be 
measured. 
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4 Determination of top mixing at large hadron 
colliders 



The lifetime of the top is too short and then, in contrast with the light 
quark flavours, its mixing can not be precisely measured studying its bound 
states. Thus, the precision with which the top properties will be known 
will depend on the ability to measure them at large colliders. The available 
top factories during the next decade will be Tevatron and the Large Hadron 
Collider (LHC) at CERN. Let us discuss in the following how the anomalous 
top couplings can be measured at these hadron machines. 

Non-dominant contributions from diagonal top vertices will be difficult 
to disentangle, not so the non-diagonal ones which can manifest in new pro- 
duction and decay processes. The analysis depends on the energy and lumi- 
nosity. The variation of the parton distributions when the energy increases 
modifies their flux and then the relevance of the processes to be consid- 
ered. On the other hand their relative statistical significance can change 
with the luminosity of the collider. Unless otherwise stated, our results will 
correspond to LHC which will also provide the highest precision. 

4.1 Production processes 

9 

t 
Q 

Figure f : Strong anomalous production of single top at hadron colliders. 

The strong FCNC in Eq. (||) can produce a single top (Fig. ([[])) |1| or 
a sine; le top plus a jet (Fig. (|)) @. These are the two lowest order strong 
anomalous processes and must then allow for the best determination of the 
new strong vertices. In the second case collisions with initial quarks (or 
with a gluon and a quark) must be also summed up. At LHC, however, the 
probability of colliding two gluons is almost an order of magnitude higher 
than that of two quarks. These vertices can also produce events with a 
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Figure 2: Strong anomalous production of single top plus a jet at hadron 
colliders. 
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Table 1: Reachable bounds on the strong anomalous couplings (Eqs. (HJH)) 
at LHC with an integrated luminosity of 10 fb _1 . The first two lines are 
divided by v2 to correct for the use of a different statistics. 



single top plus a Z boson or a photon (Fig. (D) 0. In Table | we gather 
the bounds on the strong anomalous couplings derived from these processes 
if no signal is observed. These limits give also an indication of the precision 
which can be reached in each case. 

The lowest order process involving the weak and electromagnetic FCNC 
in Eq. (Q) is Vt production, V = Z,j, but in this case with a standard 
(anomalous) strong (electroweak) vertex (Fig. (H)) []l7j. The top and the Z 
boson both decay before detection. Then we must specify their decay modes 
to define the sample we are interested in. The most significant Zt {"ft) chan- 
nel is l + l~lvb (p/lvb), I = e,n, because although it has a small branching 
ratio 1.5% (21.8%), it has also a small background ZWj (jWj). In Table 
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Figure 3: Strong anomalous production of single top plus a Z boson or a 
photon at hadron colliders. 




Figure 4: Electroweak anomalous production of single top plus a Z boson 
or a photon at hadron colliders. 



^ we collect the corresponding bounds if no signal is observed. At Teva- 
tron vvjjb is more significant because in this case the expected number of 
events is smaller and this channel has a larger branching ratio 13.6% and 
the backgrounds Zjjj, Wt, tt are not very large. 

Let us see now in an example, pp —* Zt —* l + l~lvb, how these limits are 
derived [jl?]]. To estimate the corresponding number of events we calculate 
the exact 2 — ► 5 (gq — > Zt — > l + l~lvb) squared amplitude and use it to 
generate the events with the correct distribution. Afterwards we smear 
the lepton and jet energies and apply the trigger and detector cuts to the 
resulting sample to mimic the experimental set up. 6-tagging is also required. 
Then we reconstruct the Z — > and t — > Ivb invariant masses M r <£ c and 
m r t ec , respectively (see Fig. (|5|)), and apply the kinematical cuts on m[ ec , 

(the Z tranverse momentum) and Ht (the total transverse energy). In 
this case signal and background have the same M£ ec distribution and we do 
not gain anything cutting on this variable. The number of selected events 
is given in Table [|. Finally we use the adequate statistics [19] to derive the 
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Xtu Xtc 



Ktu <Hc At u Xtc 



pp^> Zt-* l+riub 0.022 0.045 0.014 0.034 

pp^jt^ jlub 0.005 0.013 

Table 2: Most stringent bounds on the electroweak anomalous couplings 
(Eqs. (|l],||)) at LHC with an integrated luminosity of 10 fb _1 . 
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gu -> zt (y) 
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gc - ( 7 


1.1 


1.1 


5 u -> Zt (ct^) 


11.1 


10.9 


#c -► Zt (a^) 
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1.9 


ZWg u 


4.9 


1.4 


ZWq d 


5.5 


1.4 


ZWg 


4.7 


1.1 



Table 3: Number of / + / Ivb events before and after kinematical cuts 
(150 GeV < m r t ec < 200 GeV and 200 GeV < H T ) for the Zt -> l+l'lvb sig- 
nals and backgrounds at LHC with an integrated luminosity of 10 fb~ . 7^ 
and a^ v stand for the corresponding non-zero anomalous vertex (Eqs. (0j2|)). 
In this simulation we take Xt q = 0.02 and Kt q = 0.02 for easy comparison. 

expected bounds if no signal is observed. 
4.2 Decay processes 

Up to now we have assumed that the top decays as in the SM (diagram 
(a) in Fig. (||)), considering only the anomalous couplings in the produc- 
tion process. This is a good approximation because t decays predominantly 
into Wb. However, the large number of tt pairs produced at future hadron 
machines (two gluon collisions (Fig. (fj])) stand for 90% of the total cross 
section and two quark collisions for the 10%) allows also to derive competi- 
tive bounds on top mixing if no anomalous t decay is observed (diagram (b) 
in Fig. @) |||1|. 
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Figure 5: Reconstructed top mass m r t ec 7 Z transverse momentum and 
total transverse energy Ht distributions before kinematical cuts for the 
gu — > l + l~lub signals and background at LHC with an integrated luminos- 
ity of 10 fb _1 . 7^ and stand for the corresponding non-zero anomalous 
vertex (Eqs. (|l|j@))- In this simulation we take X tu = 0.02 and K tu = 0.02 
for easy comparison. 



W V 
(a) (b) 

Figure 6: Standard (a) and anomalous (b) top decay. 




Figure 7: gg — ► tt production. 

5 Conclusions 

We have reviewed present direct and indirect limits on top mixing, empha- 
sizing that this can be large in simple SM extensions with vector-like quarks. 
Future hadron colliders will reduce these bounds to the per cent level if no 
anomalous signal is observed. e + e~ colliders will also set comparable limits 
searching for e + e~ — * tq (Fig. (|8|)) and anomalous top decays [p0| . 
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Figure 8: Electroweak anomalous production of single top plus a jet at e + e 
colliders. 



References 

[1] S.L. Glashow, in Hadrons and their interactions, Proceedings Int. 
School of Physics 'Ettore Majorana', ed. A. Zichichi, Academic Press, 
New York, 1967; S. Weinberg, Phys. Rev. Lett. 19, 1264 (1967); A. 
Salam, in Elementary particle physics (Nobel Symp. No. 8), ed. N. 
Svartholm, Almqvist and Wilsell, Stockholm. 

[2] S.L. Glashow, J. Iliopoulos and L. Maiani, Phys. Rev. D2, 1285 (1970). 

[3] C. Burgess and H.J. Schnitzer, Nucl. Phys. B228, 464 (1983); C.N. 
Leung, S.T. Love and S. Rao, Z. Phys. C31, 433 (1986); W. Buchmiiller 
and D. Wyler, Nucl. Phys. B268, 621 (1986); R.D. Peccei, S. Peris and 
X. Zhang, Nucl. Phys. B349, 305 (1991); R. Escribano and E. Masso, 
Nucl. Phys. B429, 19 (1994); see also S. Bar-Shalom and J. Wudka, 
hep-ph / 9905407 and references there in. 



[4] F. Abe et al, Phys. Rev. Lett. 80, 2525 (1998); see also T. Han, K. 

Whisnant, B.-L. Young and X. Zhang, Phys. Lett. B385, 311 (1996). 
[5] J. A. Aguilar-Saavedra, Ph D Thesis. 

[6] N. Cabibbo, Phys. Rev. Lett. 10, 531 (1963); M. Kobayashi and T. 

Maskawa, Prog. Theor. Phys. 49, 652 (1973). 
[7] C. Caso et al, European Phys. Journal C3, 1 (1998). 
[8] Y. Nir and D. Silverman, Phys. Rev. D42, 1477 (1990); D. Silverman, 

Phys. Rev. D58, 095006 (1998). 
[9] F. del Aguila, J. A. Aguilar-Saavedra and R. Miquel, Phys. Rev. 



Lett. 82, 1628 (1999); F. del Aguila and J.A. Aguilar-Saavedra, [hep 
ph/9906461. 



14 



[10] D. Karlen, in Proceedings of the International Conference on High En- 
ergy Physics '98, Vancouver, 1998. 

[11] G.F. Tartarelli, CDF Collaboration, in Proceedings of International 
Europhysics Conference on High Energy Physics (HEP 97), World Sci- 
entific, 1997. 

[12] P. Abreu et al, Phys. Lett. B446, 62 (1999); DELPHI note 99-85; 
DELPHI note 99-146. 

[13] T. Han, R.D. Peccei and X. Zhang, Nucl. Phys. B454, 527 (1995). 

[14] F. del Aguila and M.J. Bowick, Nucl. Phys. B224, 107 (1983); G.C. 
Branco and L. Lavoura, Nucl. Phys. B278, 738 (1986);. P. Langacker 
and D. London, Phys. Rev. D38, 886 (1988); E. Nardi, E. Roulet and 
D. Tommasini, Nucl. Phys. B386, 239 (1992);. V. Barger, M.S. Berger 
and R.J.N. Phillips, Phys. Rev. D52, 1663 (1995); P.H. Frampton, 
P.Q. Hung and M. Sher, [hep-ph/9903387| , Phys. Rep. (in press) and 
references there in. 

[15] M. Hosch, K. Whisnant and B.-L. Young, Phys. Rev. D56, 5725 (1997). 

[16] T. Han, M. Hosch, K. Whisnant, B.-L. Young and X. Zhang, Phys. 
Rev. D58, 073008 (1998). 

[17] F. del Aguila, J. A. Aguilar-Saavedra and LI. Ametller, Phys. Lett. 



B462, 310 (1999); F. del Aguila and J.A. Aguilar-Saavedra, [hep 
ph/9909222} 



[18] T. Han, K. Whisnant, B.-L. Young and X. Zhang, Phys. Rev. D55, 
7241 (1997). 

[19] G.J. Feldman and R.D. Cousins, Phys. Rev. D57, 3873 (1998). 

[20] T. Han and J.L Hewett, Phys. Rev. D60, 074015 (1999); see also V.F. 
Obraztsov, S.R. Slabospitskii and O.P. Yushchenko, Phys. Lett. B426, 
393 (1998). 



15 



